Critic-Proofing: Robust Validation Through Data-Mining

Critic-proofing is a modified heuristic evaluation technique, specifically designed to provide a fine-grained, prioritized list of heuristic violations. The critic-proofing technique weights the severity of a problem based on the frequency that similar problems are found in similar games. The severity ratings are calculated using data collected from game reviews, and the severity assigned to a problem during the heuristic evaluation process. However, heuristic techniques have had limited adoption within the video game industry. One reason for this is the perceived lack of validity and robustness of game specific heuristic principles. In this paper, we introduce and outline a new datamining project designed to validate game-specific heuristic techniques, especially the critic-proofing technique by using the popular game-review aggregation website Metacritic.